Skip to content

feat(storage): OpenTelemetry client metrics instrument registry and views - #14398

Open
nidhiii-27 wants to merge 17 commits into
mainfrom
feat/storage-client-metrics-infra
Open

nidhiii-27 wants to merge 17 commits into
mainfrom
feat/storage-client-metrics-infra

Conversation

@nidhiii-27

@nidhiii-27 nidhiii-27 commented Sep 16, 2026 •

Copy link
Copy Markdown
Contributor

This PR implements Phase 1 (Core Metrics Registry & Views) of OpenTelemetry Client Metrics in the Java Cloud Storage SDK (google-cloud-storage).

Following review feedback and architectural alignment across Storage SDKs, this PR focuses strictly on the package-private instrument registry and OpenTelemetry view/histogram bootstrapping, decoupling core infrastructure from public configuration options (handled in follow-up PR #14484).

Summary of Changes

  • Instrument Registry (StorageClientMetrics):

    • Implemented package-private registry defining standard and debug OpenTelemetry instruments (call/request durations, operation and retry counters, transfer sizes, TTFB, active requests, stall duration, etc.).
    • Configured standardized attribute keys and UCUM unit strings (e.g., s, By, {operation}, {attempt}, {retry}).
  • OpenTelemetry Views & Exporter Bootstrapping (OpenTelemetryBootstrappingUtils):

    • Registered custom explicit-bucket histogram boundaries for latency (2ms resolution) and size distributions (128KiB to 16GiB).
    • Configured metric name view translation (. to /).
    • Added createClientMeterProvider to bootstrap Google Cloud Monitoring export under custom.googleapis.com with client attributes (instance_id, api).
  • Testing:

    • Added unit test suites (StorageClientMetricsTest, OpenTelemetryBootstrappingUtilsTest) verifying metric registrations, custom view bucket distributions, and exporter bootstrapping using InMemoryMetricReader.

[Generated-by: AI]

@nidhiii-27
nidhiii-27 requested review from a team as code owners September 16, 2026 09:21

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces OpenTelemetry client metrics support for Google Cloud Storage, allowing standard and debug metrics to be configured and recorded for both HTTP and gRPC transports. It adds configuration options to StorageOptions, GrpcStorageOptions, and HttpStorageOptions for enabling metrics, setting custom meter providers, and defining export intervals, supported by a new StorageMetricsConfig utility and StorageClientMetrics registry. Feedback on the changes suggests importing java.time.Duration to avoid using fully qualified class names throughout GrpcStorageOptions.java and OpenTelemetryBootstrappingUtils.java, which will improve code readability and consistency.

@nidhiii-27 nidhiii-27 added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Sep 16, 2026
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Sep 16, 2026
@nidhiii-27 nidhiii-27 changed the title feat(storage): OpenTelemetry client metrics infrastructure and configuration feat(storage): OpenTelemetry client metrics instrument registry and views Sep 23, 2026
@nidhiii-27
nidhiii-27 added this pull request to stack #14485 September 23, 2026 10:36
@nidhiii-27 nidhiii-27 added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Oct 7, 2026
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Oct 7, 2026

@cpriti-os cpriti-os left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes mainly for the metric types/names and for accuracy of the recorded data (see inline comments).

One question on the overall design: createMeterProvider is only reached from enableGrpcMetrics(). Please state in the PR description how metrics are exported for HttpStorageOptions when metrics are enabled and no MeterProvider is supplied; both transports are expected to export through the same SDK-owned provider.

@nidhiii-27
nidhiii-27 requested a review from cpriti-os October 8, 2026 07:29

@cpriti-os cpriti-os left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for addressing the previous comments! Left a few small updates from our latest metrics/attributes alignment.

@Dhriti07
Dhriti07 self-requested a review October 16, 2026 08:23
…nfiguration

- Add configuration options to StorageOptions, GrpcStorageOptions, and HttpStorageOptions (enableOtelMetrics, enableOtelDebugMetrics, meterProvider, metricInterval).
- Add development gate checking system properties and environment variables in StorageMetricsConfig.
- Implement package-private instrument registry in StorageClientMetrics.
- Update OpenTelemetryBootstrappingUtils to register client views with custom histogram boundaries and create client meter provider.
- Add comprehensive unit tests in StorageOptionsTest, StorageClientMetricsTest, and OpenTelemetryBootstrappingUtilsTest.
Apply spotify fmt-maven-plugin formatting to StorageMetricsConfig, StorageClientMetrics, StorageOptionsTest, and StorageClientMetricsTest.

[Generated-by: AI]
Use simple class name Duration instead of fully qualified java.time.Duration in GrpcStorageOptions and OpenTelemetryBootstrappingUtils.

[Generated-by: AI]
…trappingUtils

Apply spotify fmt-maven-plugin formatting to OpenTelemetryBootstrappingUtils.

[Generated-by: AI]
… metrics

Refactor addClientHistogramView in OpenTelemetryBootstrappingUtils to remove unit selector and custom view description, preserving instrument default descriptions.

Remove deferred Phase 2 connection metrics (dnsLookupDuration, tcpConnectDuration, tlsHandshakeDuration) from StorageClientMetrics, CLIENT_LATENCY_HISTOGRAMS, and StorageClientMetricsTest.

Ensure clean Duration imports.

[Generated-by: AI]
…rProvider

Default api parameter to "grpc" instead of "storage" in createClientMeterProvider, allow specifying "json" / "http" transport protocols for Cloud Monitoring monitored resource storage.googleapis.com/Client, and add unit test coverage in OpenTelemetryBootstrappingUtilsTest.

[Generated-by: AI]
Update createClientMeterProvider to use prefix custom.googleapis.com with default monitored resource mapping.
Add KEY_URL_TEMPLATE attribute key constant in StorageClientMetrics.

[Generated-by: AI]
@Dhriti07
Dhriti07 force-pushed the feat/storage-client-metrics-infra branch from 910b46d to d66f432 Compare October 20, 2026 05:37

@Dhriti07 Dhriti07 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for addressing the comments ! LGTM, Holding on approval due to some failing storage tests due to unrelated issues from this PR, once the Integration test pipeline is working again and is green we can unblock this PR

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants